Creating Multifiles¶

Multifiles archives are archive files that store game resources. Think of it as a giant zip file that stores, optionally zips and encrypts your data files, but does not need to be extracted.

The multify program¶

The multify console program creates such files. You can get information about the commandline parameters by running multify with the -h option. This is how program describes itself:

Usage: multify -[c|r|u|t|x] -f <multifile_name> [options] <subfile_name> ...

multify is used to store and extract files from a Panda Multifile. This is similar to a tar or zip file in that it is an archive file that contains a number of subfiles that may later be extracted.

Panda’s VirtualFileSystem is capable of mounting Multifiles for direct access to the subfiles contained within without having to extract them out to independent files first.

The command-line options for multify are designed to be similar to those for tar, the traditional Unix archiver utility.

Read Assets¶

If you want to prepare to read assets from a Multifile directly, you can “mount” it into the virtual file system:

from panda3d.core import VirtualFileSystem
from panda3d.core import Multifile
from panda3d.core import Filename

vfs = VirtualFileSystem.getGlobalPtr()
vfs.mount(Filename("foo.mf"), ".", VirtualFileSystem.MFReadOnly)

If you want to read assets, you can mount a whole directory structure from a webserver.

If your webserver hosts:

http://localhost/mydir/models/myfile.bam
http://localhost/mydir/maps/mytexture.png

Put this in your config.prc:

vfs-mount-url http://myserver/mydir /mydir
model-path /mydir

Or, equivalently, write this Python code at startup:

vfs.mount(VirtualFileMountHTTP('http://myserver/mydir'), '/mydir', 0)
getModelPath().appendDirectory('/mydir')

and then you can load models like this in your Python code:

model = loader.loadModel('models/myfile.bam')
texture = loader.loadTexture('maps/mytexture.png')

If you want to prepare for reading and writing assets to a Multifile do the following.

from panda3d.core import VirtualFileSystem
from panda3d.core import Multifile
from panda3d.core import Filename

mf = Multifile()
mf.openReadWrite("models.mf")

vfs = VirtualFileSystem.getGlobalPtr()
if vfs.mount(mf, ".", VirtualFileSystem.MFReadOnly):
  print 'mounted'

If you want to prepare for reading and writing assets to a ‘subdirectory’ Multifile do the following. Note “mysys” must always be literally written in any python code. E.g. “mysys/memfdir/mfbar2.txt”

from panda3d.core import VirtualFileSystem
from panda3d.core import Multifile
from panda3d.core import Filename

mf = Multifile()
mf.openReadWrite("models.mf")

vfs = VirtualFileSystem.getGlobalPtr()
if vfs.mount(mf, "mysys", VirtualFileSystem.MFReadOnly):
  print 'mounted'

If you are having problems loading from multifiles you can list the complete contents of your .mf file with a command like:

multify -tvf mymultifile.mf

Doing a sanity inspection like this can be useful to ensure that your assets are in the right place within the multifile.

Multifile objects¶

The Multifile class is designed for opening, reading and writing multifiles. You can open a new multifile by creating an instance of the class and calling the openRead method:

from panda3d.core import Multifile

mf = Multifile()
mf.openRead("foo.mf")

The openRead method opens the multifile as read-only. If you want to make changes to it and write it back to disk, you will need to use the openReadWrite method. Also, there exists openWrite to create a new multifile.

If you have made important structural changes to a Multifile, it is recommended to rewrite the multifile using the repack() method. (This won’t work if you’ve opened it using openRead.) If you are uncertain about whether it has become suboptimal, you can call needsRepack() which returns True if the Multifile is suboptimal and should be repacked.

To write it back to disk, you can use the flush() method which flushes the changes you’ve made to the multifile back to disk, or the close() method if you’re done with the file.

To mount Multifile objects into the VirtualFileSystem without writing them to disk first, here’s an example on how to mount them:

yourMF = Multifile()
#... now do something with yourMF

vfs = VirtualFileSystem.getGlobalPtr()
vfs.mount(yourMF, ".", VirtualFileSystem.MFReadOnly)

Subfiles¶

Files that are added to a multifile are called subfiles. You can add existing files to a multifile object using the addSubfile function. This function takes three arguments: the target filename, the existing source file and the compression level (1-9). There is also updateSubfile, which does the same thing but if the file already exists, only updates it if the content is different.

There are several other methods which operate on subfiles, which you can find in the API Reference. Here are a few examples of working with subfiles:

from panda3d.core import VirtualFileSystem
from panda3d.core import Multifile
from panda3d.core import Filename

m = Multifile()

# Add an existing real os file with compression level 6
m.openReadWrite("foo.mf")
m.addSubfile("bar.txt", Filename("/tmp/bar.txt"), 6)
m.flush()

# Destroy the contents of the multifile
# Add an existing real os file to be the first multifile
m.openWrite("foo.mf")
m.addSubfile("bar.txt", Filename("/tmp/bar.txt"), 6)
m.flush()

# Permanently re-order in ascending order the
# directories and files in the multifile
m.openReadWrite("foo.mf")
m.repack()
m.flush()

# Open a multifile and replace the contents of the mulifile file
# with new contents
m = Multifile()
m.openReadWrite("foo.mf")
m.updateSubfile("bar.txt", Filename("/tmp/bar2.txt"), 9)
m.flush()

# Open a multifile and extract all files smaller than 3kb
# New real os files are created with the contents of the multifile data
m = Multifile()
m.openRead("foo.mf")
for i in range(m.getNumSubfiles()):
  if m.getSubfileLength(i) < 3 * 1024:
    m.extractSubfile(i, Filename("/tmp/" + m.getSubfileName(i)))

# Find, print and remove a file named bar.txt
barIdx = m.findSubfile("bar.txt")
if barIdx != -1:
  # It returns -1 if it doesn't exist
  print m.readSubfile(barIdx)
  m.removeSubfile(barIdx)
m.flush()

m.close()

If the foo.mf file were to have a contained bar.egg.pz file, load the egg and use it similar to other model loading methods.

nodepath = loader.loadModel("foo/bar")

Stream-Based¶

Multifile algorithms are stream-based and not random-based. In a running game, from the output, if a message is received saying something similar to the words seek error for offset then a file in the multifile is trying to be accessed by a random-based method. For multifiles and fonts, an example of a random-based file is an .rgb file. An alternative different from the use of an .rgb file is the use of a .ttf instead. An example follows.

# models is the original directory
# models.mf it the new target multifile
multify -c -f models.mf -v models

In the game, from the multifile models.mf, load the .ttf file.

font = loader.loadFont("models/arial.ttf")

Encryption¶

Multifiles can also encrypt your files with a password. To do so, you need to set the encryption flag and password using the setEncryptionFlag and setEncryptionPassword methods, before adding, extracting or reading multifiles.

At the OS prompt, to create a password protected multifile and print out the contents do the following.

# models is the original directory
# models.mf it the new target multifile
multify -c -f models.mf -ep "mypass" -v models

This code creates a multifile and adds an encrypted file to it:

m = Multifile()
m.openReadWrite("foo.mf")
m.setEncryptionFlag(True)
m.setEncryptionPassword("foobar")

# Add a new file to the multifile
m.addSubfile("bar.txt", Filename("/tmp/bar.txt"), 1)
m.flush()
m.close()

You can read encrypted multifiles the same way:

m = Multifile()
m.openRead("foo.mf")
m.setEncryptionFlag(True)
m.setEncryptionPassword("foobar")
# Prints the contents of the multifile
print m.readSubfile("bar.txt")

At the OS prompt, to see the contents of a password protected multifile perform

multify -tvf models.mf -p "mypass"

You can test the reading in a of password-protected multifile, followed by the mounting of the file using the following code.

from panda3d.core import Multifile
mf = Multifile()
mf.openRead("models.mf")
mf.setEncryptionFlag(True)
mf.setEncryptionPassword("mypass")

from panda3d.core import VirtualFileSystem
vfs = VirtualFileSystem.getGlobalPtr()
if vfs.mount(mf, ".", VirtualFileSystem.MFReadOnly):
   print 'mounted'

When running the game, the following should be seen.

mounted

You can check if a certain subfile is encrypted or not using the isSubfileEncrypted method, which takes the subfile index as parameter.

It is possible to have a multifile where different subfiles have different encryption, but you will not be able to mount it with the VirtualFileSystem or use it with the multify tool. To mount an encrypted file using the VirtualFileSystem, pass the password as parameter to the mount method:

from panda3d.core import VirtualFileSystem, Filename
vfs = VirtualFileSystem.getGlobalPtr()
vfs.mount(Filename("foo.mf"), ".", vfs.MFReadOnly, "foobar")

To use encryption with the multify tool, run it with the -e option, which will prompt for a password on the command line. Alternatively, if you also specify the -p "password" option, you can specify it in the command instead of typing it at the prompt.